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Designed to determine how well the student materials 
and teachers' guides of commercial spelling programs inform teachers 
of and reflect important spelling research, present management 
systems adaptable to individual student needs, provide strategies 
eliminating unnecessary or nonspelljng instruction, and maximize the 
provisions for individual student differences, the Spelling Program^ 
Effectiveness Rating Scale was applied to 10 nationally known 
spelling programs and two programs designed on research-supported 
strategies. Evaluation results implied that (1) programs with 
research-supported and nonresearch-supported strategies differed 
widely; (2) teachers' guides generally failed to report on the 
literature of spelling, the criteria for word selection, or any 
empirical research on the effectiveness of their programs; (3) most 
manuals did not provide clear procedures for handling individual 
student differences; (4) student materials contained a wide variety 
of word study activities that were not supported as spelling 
strategies; and (5) only three of the evaluated programs received an 
overall rating that was higher on the positive scale than the 
negative scale. (MM) 
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INTRODUCTION 

While spelling research is one of the more heavily 
researched areas of the* curriculum, there is an obvious 
lag in making the results of such research available to the 
classrrom practitioner (Horn r 1944; Fitzgerald, 1951; 
Campanela, 1962). If the reporting of research is limited to the 
professional journals and authors tend to avoid discussing 
research in the teachers guides of their published materials, 
it is possible for spelling programs to be selected * by state 
textbook adoption committees and by local school districts 
without knowing the important relationship between the programs 
under consideration and the theoretical and empirical literature. 

The purpose for the development of the Spelling Prograln 
Effectiveness Rating Scale — SPERS — was three fold. First, 
it wcpuld provide a guide to evaluate spelling programs on some 
bases other than quality of the bindings, methods used to. disguise 
the grade level designations, scope and sequence, ease of 
teacher use, and the number of remedial and enrichment activites 
included. Second, because those in the position of evaluating and 
selecting spelling programs may not be knowledgable of strategies 
that are supported or not supported in the literature, the SPERS 
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ight provide information which would heighten the evaluator's 
areness of such literature. Finally, the use of a rating scale 



such as the SPERS v^puld provide educators with a procedure for 
comparing- programs with each other and with programs that 
represent high as well as low ratings on the Scale. 

DEVELOPMENT OF THE SPERS 

Two questions- were developed initially and used as guide- 
lines for the items selected in the SPERS. 

1. What information is included in a teacher's guide to 
a spelling program that: 

a) informs teachers of the theoretical and empirical 
literature on spelling instruction ; 
*• b) reflects instructional* strategies that are 

supported by the research; and 

■ T 

c) provides for* management systems that can be 
clearly followed and that will accommodate the 
individuality of t K e spelling process as well 
x as the individual needs of students? 
2c What strategies and procedures are included in* the 
student materials that: 

a) reflect instruction that is supported or not 
supported in the literature; 

b) eliminate unnecessary , time consuming 9 or 
non-spelling instruction; and 

c) maximizes the provisions for individual differences? 
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Using the two questions as guidelines, 36 declarative 
statemeots w£re developed — 12 for the' teacher 1 s guide and 
24 for the 1 materials used by the students. Each statement can 
be answered by circling* a YES or NO in the columns vo the 
right of the statement (Appendix A ) . If a YES response to 
an item is either supported by the literature or is a 
desireable attribute , the YES will appear in the Positive 

Scale column. On the other hand if a YES response to an 

« 

item is considered to be a nonsupported strategy or a non- 
spelling activity, the YES will appear in the Negative 

Senile column. For example a response of YES to statement 

a * • 

number 1 in Part II would result in a positive score 'for 
that* program since research Clearly supports the use of 
a pretest at the beginning of a lesson and prior to 
any instruction. On the other hand a response of YES to 
statement number 9 in Part II would result in a negative 
score for that program becausejthe practice of merely 
writing spelling words three or four times without self-** * * 
correction after each writing of the word has little or no 
support in the literature. 

By responding to the 12 statements in Part I and the 
24 statements in Part II, the evaluator can obtain a positive 
as well as a negative rating on (1) the manual of instructions 

c * 

(2) the materials used by the student, and (3) the overall 
program. Each item is awarded two points. The maximum 
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rating is 24 on Part I, - 48 on Part II, or a total of 72. 

EVALUATION RESULTS OF 12 SPELLING PROGRAMS 

Twelve spelling programs were selected for the study. 
Ten programs were selected because they were considered to 
be nationally known, had a large number of sales , and were 
usually submitted to state textbook adoption committees for, 
consideration. Two rather obscure programs were also selected 
because they claimed to be designed strictly on research- 
supported strategies — Curriculum Associates and Educational 
Services. 

The positive ratings , negative ratihgs, and overall 
ratings for each of the 12 programs are illustrated in Table 

4 

1. It should be noted that the two obscure programs identified 
above received the highest ratings in all three categories. 
The one program believed to have the largest number of sales 
(not substantiated) received the lowest ratings in all three • 
categories. The 12 programs seem tip be clustered! into three- 
groups , particularly when only the students materials "are 
taken in£o consideration. The first cluster of two programs 
(Curr. Assoc. and Educ. Services) have ratings substantially 
higher than all other programs — in all three categories. 

The second cluster of fiVe programs (Follett r HBJ, Laid- 
law r ABC, and Scott Foresman) have positive ratings between 24 

\ 

- o 
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and 23 (X = 25.6) on the student materials and a range of 20 * 
to 24 (X = 22.4) on the Negative Scale. In th^is group of 
programs there were nearly as many negative as there were 
positive features with regard to the student materials. On the, 
other hand these same programs rated poorly on the teacher's 
guides with a mean positive rating of 5.2 and a mean negative 
rating of 18.8. 

The third cluster consists of five programs receiving 
significantly- higher scores on the Negative Scale than on the. 
Positive Scale (Economy , Harper and Row, Silver Burdett, Rand 
McNally, and t McGraw-Hill) . The range of sfrg^res on the Positive 
Scale was 8 to 18 (X = 15.6) while the range was 30 to 40 
(X = 32.4) on the Negative Scale — when considering the 
student materials. For an item-by-item analysis refer to 
Table 2. . 

DISCUSSION , ■ 

If the SPERS reflects a realistic approach to the 
evaluation of spelling programs, the following implication 
might be drawn: 

1 . There appears to be wide differences among programs 
when research supported and nonresearch supported strategies, 
are considered. ; 

2. Teacher's guides for the most part fail to report 
on the literature of spelling , the criteria used in the 

^election of words -for the various levels of their programs. 
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and the emiprical research thfet may have been done on the 
effectiveness of their programs. About the only research 
reported is that related to varification studies and other 
field work used to work out "bugs" in the program. This , % 
varifies Fitzgerald's (1951) claim* that "results of research 
and experimentation were not readily available to the 
teacher (p. 2-3) . " 

3. Although publishers will discuss making provisions 
for individual differences among students and recognize its 
importance , they continue to leave out clearly delineated 
procedures that a teacher can follow to provide for individual 
differences such as the. use of initial placement tests , pre- 
testing for each lesson prior to instruction, self-pacing , 
periodic recycling through lessons when needed, and models of 

t* i r 

alternative program management to enable teachers to use 
ability grouping, peer-teaching, and individual self-study. 
In reality most manuals, by their superficial treatment of 
individual differences, encourage a 5-day lesson cycle, 
whole class instruction, disregard for previous knowledge 
of the vocabulary, and the study of all words by all students 
whether needed or not. 

4. Student materials seem to contain a wide variety of 
"fluff" activities involving word study. While these activities 
may by useful in* a broader "language arts" sense, few of them 

c 

are clearly supported as spelling strategies. Most programs 
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c^ptinue to encorporate practices such as linguistic groupings 
(sound or letter patterns) , word definitions f cloze exercises , \ 
dictionary study , and the learning of spelling generalizations 
as well as exceptions — all of which are non-spelling activities. 
The most clear-cut strategy of all — Fitzgerald's (1951) 
look, say, visualize, write, and self -correct — is usually 
explained on the first page of the student's book but never 
referred to again neither in the teacher*s guide nor in the 
student's material. 

5. Only three programs evaluated received a higher 
overall rating on the positive scale than on the negative 
scale (Curriculum Associates, Educational Services, and 
Jollett). , . . 

This might be interpreted to mean that the more widely- 
used programs spend considerably more time with non-spelling 
instruction than with instruction that is considered to 
improve spelling achievement. Perhaps Christine and 
Hollingsworth (1966) were correct when th^y suggested that 
proper spelling strategies could mean that students could 
achieve the same results in half the time. 

i - • 

CONCLUSIONS ~ ' 

Petty stated "that much has been learned but the 
knowledge has not been used (1969., p. 79). Christine and 
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Hollingsworth (1 966)* made a similar observation when they 
wrote that "Many studies concerning spelling instruction 
have been made in the twentieth century, yet many pupils 
Have learned to spell incorrectly (p. f>65) w • 

We have often wondered whether the lag between 
research and practice could be attributed to publishers 1 
concern for sales rather than for what is truth* Perhaps 
the availability of the SPERS will help to close this gap 
by (1) providing a means of evaluating programs based on 
the. literature rather than on packaging, (2) educating teachers 
through this evaluation process, and (3) raising '"spelling 
literacy because of 'a more, logical s/slection of instructional 
materials. * 
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TABLE 1, MEAN SCORES OF 12 SPELLING PROGRAMS IN THE THREE CATEGORIES 
OF THE SPELLING PROGRAM EFFECTIVENESS RATING SCALE - SPERS 





TEACHER'S GUIDES 


STUDENT MATERIALS 


COMBINED RATING 


SPELLING PROGRAM 


POSITIVE 


NEGATIVE 


POSITIVE 


NEGATIVE 


POSITIVE 


NEGATIVE 




CPAI C 


CPAI E 


OTA! C 

SCALE 


SCALE 


SCALE 


SCALE 


EDUCATIONAL SERVICES 


22 


2 


18 


' 0 ' 


: 70 


2 


CURRICULUM ASSOCIATES 


20 


• 1 


10 


8 


60 


12 


FOLLETT 


10 


\11 


28 


20 


38 


31 


HARCOURT BRACE 


1 


20 


26 


22 


30 


12 


i A t ni a iii 

LAIDLAW 


2 


22> 


26 


22 


28 


11 


AMERICAN BOOK 


1 


20 


21 


21 


28 


11 


SCOTT FORESMAN 


6 


18. 


21 


21 


30 


12 




h 
H 




Is 


?n 
)U 


22 


50 


HARPER' AND ROW 


11 


10 


18 


30 


32 


10 


SILVER BURDETT 


2 


22 


18 


30 


20 


52 


RAND MORALLY 


1 ' 


20 


16 


32 


20 


52 


MCGRAW-HILL 4 


2 


22 


, 8 


40 


10 


62 


' MEAN SCORES 


' 7,8 


16.2 


•21,5 


23,5 


32,3 


39,7 



(1) Programs are arranged in rank order based on the Positive Scale 
For Student Materials. 

(2) Maximum score for Teacher's Guides = 24 

(3) Maximum score for Student Materials = 48 

(4) Maximum score for Combined Rating = 72 



TABLE 2. NUMBER OF RESPONSES ON THE POSITIVE AND NEGATIVE 
SCALE FOR fiACH OF THE 36 ITEMS ON THE SPERS (N = 12) 



Description of Items on the SPERS 


Positive 
Scale 
N=12 


Negative 
Scale 
N=12 


Teacher's Guide -12 Items 
i • clues researcn on spelling 


ies- 2 


Mo- 9 * 
wo— 1 


uuiae conudins DiDiiograpny 


Yes- Jt 


NO— /if 


Aesearcn reportea on mau program 


Yes- ^ 


NO— fO 


4« oource or vocaouiary xaentinea 


Yes— 


NO- 


D« lii terid ror wora piacemenu laentniea 


Yes— / 


NO— // r 


o* initial placement test 


Va 

Yes— ^ 


NO— | ~ 


/ • lnoti v Liuiio on lnvji viuuaiizing 


Yes- 3 


NO— 7 


8- Periodic fp^fina of lona-term memorv 


Vpes 

x Co ^ 


No= V 


Q Mi^cjnpllpH wnrH? carripd forward for studv 

^ m LX X O O V* w J. X CU W V X VI O V» CI XXX w VI X V/ X ww O X VI X V/ X w UUUJ 


x Ca~ 7 




10. Charts matching rules w/ specific lessons 


' No= fT 


Yes= 7 


11. Alternative management strategies 


Yes= y 


No= 


12. Suggestions on time spent in spelling 


' Yes= 


No= 


MEAN RESPONSES ON TEACHER'S GUIDE 


«A p 1 7 


X = 1.1 


* 

student Materials — i terns 
1 • ricLcSL pr lot to »ny inotxuv.tiv/n 


Vo c = *r 

Z Co— ^ 


Nr»= 7 


2-. WotHq Dfps^nf pH ifi column form 

X • FfV/XVIO k^Xww^rtl LwU XA1 vw^Ulllll XV/XI1I 


Yes= 


No= ^ 


3. Words grouped . by sound or letter patterns 


No= / 


Yes« //* 


4. Words grouped by frequency of use 


Yes= 3 


No= 7* 


5. "Look-say-write-correct" is explained 


Yes* /tf* 


No* % 


6. "Look -say-write" is found in all lessons 


Yes*. 5* 


N6* 7 
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TABLE 2 (Continued) 



Description of Items oh the SPERS 


Positive 
Scale 
N=12 


Negative | 
Scale 
N=12 


7. Concentrate only on misspelled words 


Yes= 3 


No= 9 * 


8. Rules stated/implied in each lesson 


No= % 


Yes= JO* 


9. Write words 3 or more time 


No= /£* 


Yes= £? 


10. Regular written dictation-self correction 


Yes= £ 


No= £ 


11. Wd. visualizing activities 


Yes= </ 


No= f * 


12. Meanings of words, prefixes, suffixes, etc. 


No= 3 


Yes= <y * 


13. Clpze activities, filling in blanks 


No= % 


Yes= fo* 


14. Words written in isolation on final test 


Yes= ft 


No= £ 


15. Sentence writing on final test 


no= 


Yes= 3 


16. Trans, dictionary spelling on final test 


• 

no= ir 


Yes= / 


17. Writing wds. in sent. /creative writing 


Y ? s = f 


No= 7 


18. Studying "hard spots" in words 


No^ f& 


Yes= 1 


19. Lessons discuss exceptions to rule in the lesson 


No= 7 


Yes= /" 


20. Translate non-traditional to tr^itional spelling 


No= //* 


Yes» / 


21. Handwriting-letter formation practice 


No= S 


Yes* 7 


22. Dictionary practice 


No= 3 


Yes» 7 * 


23. Misspelled wds* carried forward and tested 


Yes= y 


No* % * 


m rVUb • dUUcU J. L Will UtilCl oUULLco 


Yes= y 


No= £ * 


MEAN RESPONSES ON STUDENT MATERIALS 






MEAN RESPONSES , ALL 36 ITEMS 


x = r.H 
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SPELLING PROGRAM EFFECTIVENESS RATING SCALE (SPERS) 
Elton G. Stetson, Ed . D. and Frances J. Boutin, Ed.D. 

Name of Program Evaluated — Level Evaluated 

Publisher Pub. Date Evaluator 



PART -I* EVALUATION OF THE TEACHER'S GUIDE 

Instructions: Using the teacher's guide or manual of 
instructions, respoi* toieach of the statements below 
by circling the YES obc Npf in the columns to the right. 



Statements Corfcerning 



1. Manual contains a r 
instructional strate 
supported by resqficch 
are not support 



Manual include 
on spelling in 





eacher's Guide 



of the literature on 
s in spelling that are 
well as those that 



ibliography of research 
uction. 



3. Manual des/r^bes [research that has been com- 
pleta^cT^>h the^^cectiveness of that program. 
Fie/d studies to work out "bugs" in the 
pr<rcram, nxhqbeft of copies sold, or number of 
editions prirrfed does not count. 



4. Manual cle 
of the voc 



ly identifies the specific sources 
ulary used in the program. 

5. Manual £Tj£ther identifies the criteria used to 
determine the specific levels of the program 
in which each vocabulary was placed. 

6. Manual contains a placement or competency test 
that is to be given at the beginning of the 
year or program to determine whether a student 
is (a) competent at that level and heeds in- 
struction at a higher level, (b) at the propor 
level/ or (c) should be placed at a lower level 



Positive 
Scale 

YES 



Negative 
Scale' 



YE 



YES 




fES^ 

YES 

YES 




fNQ 




[NO 



NO 



(2? 



Copyright, 1983 by Elton Stetson and Frances Boutin. 
Reproduction of this scale cannot be reproduced 
without the written consent of the authors. 
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Positive Negative 
Scale Scale 



7. Manual contains specific instructions on how 
to individualize the program so that students 
within the same class can work on different 
lessons or different levels of the program* 



8. Manual include 
administered pe 1 
weeks or so) th 
of words previop 
do not count. 

9. Manual provid 
to' (a) carry 
were spell inctf 
test; and (b)^st 

. [Both a and 




dictation tests to be 
(every 9 or 18 
bng- range retention 
taught. Review lessons 

ecific" instructions on how 
Spelled words forward that 
ectiy on the final lesson 
and retesting of words, 
^be included to circle. YES] 

10. Manual coAta^ns a chart identifying the many 
phoneme-gi^uaheme associations covered in the 
program a 1 onto with references to the specific 
lesson&whera instruction is available on that 
sound/^ttei| pattern. 



inu 






>vides clearly explained details on 
rmarrag inflame program in ways other than the 
traditional 3-day or 5-day lesson cycle in^ 
[volving the entire class — such as ability 

jg> peer teaching, or individual self- 
study, 

12. M^qu^C discusses the specific amount of 
time that should be spent each day or each 
week. [Research. concludes that 60-75 minutes 
per week Is optimum] 

SCORING PART I 

IA. TOTAL WORDS CIRCLED ON EACH SCALE 
{Maximum Score = 12] 



IB. TOTAL RATING FOR TEACHER'S GUIDE 
[Multiply numbers in IA by 2] 



YES 




fYES 



YES 



NO 



YES 




x 2 ' 



NO 



CP 




[YES 



(nc^ 



NO 



± 



x 2 



1% 
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PART Hi EVALUATION OF* STUDENT MATERIALS 



Instructions: Randomly select four lessons (not review 
lessons) . Read each statement below , evaluate each of 
the four lessons, and circle the YES or the NO based 
on the following criteria: 

Circle the YES if the statement "applies or can. 

be found in two or more of the four lessons. 

Circle the t NO_if Jthe Statement does not apply 
or the strategy is no A fourAin the lessons.. 

Note: Activities th 
or extra work and 
should not be cons 



a in 
, del 



A pretest; is alw 
beginning' of t 





d "optional" 
regular program 



Statements Concerning th\ Student Materials 



administered at the 
son prior to instruction. 

Words to be leirned are presented initially 
in a list or column (in isolation) and not 
contained in phrases or sentences. 



Positive 
Scale 

YES 



fES 




3. Words i 
sound p 
a conuno 




ssons all 
e.g., the 
ter pattern ( 



Ibrdl are placed together 
[according to their frequ^ 
lthan according to any so 




taki a common 
9 /I/Jioun^) or 
» t jfc^Bat , etc. 



the ,same lesson 
r use rather 
or letter pattern. 



5. At the beginning Of ther Bbok prior to the 
f irst^l^esson , instructions^ are provided to 

the student on how to j^tudXwords independently, 
[e.g., look at the wofdVsa^the word, cover 
and pronounce the wo/L cover and write the 
word f uncover and chfccii spelling] 

6. Each lesson includesl^n exercise or provides 
time for student self-study of words as 
described in numpac 1 above. 

Students a^f^taug^ to concentrate their study 
efforts ofcly on those words missed during the 
pretest richer than to study all words, 

8. Lessons tend tolbtate or imply a specific rule 
or generalization in spelling. 



NO 



YES 



ES 




YES 



YES 



NO 



9. There are specific directions for the student 
to write each word in isolation two or more 
times for practice. * 



10. At least once d 
are directed to 
teacher or peer a 
word [Both a and 



11. There are act 
student visual 
imagery thrpug 
the configurati 
looking for la 
f ixesy suf f ii 




12. There are 
word meani 1 
looking up 
mean " 




ch lesson students 
e words dictated by 
sel f-correct each 

present To say yes] 

that center on the 
e word to develop 
rivities suc h as s tudying 
f the word ncTintngl ] , 



word parts such as pre- 
anagrams. 




itilf involving the study of 
such as etiology or word origins, 
anings in the dictionary, multiple 
nings of prefixes or suffixes. 

tivities in which the student 
entences, fills in missing words 
ences^or other context clue or 
ities. 



in 

peer. 



final test^or efefifc^lesson, the 

writes the v^ords in isolation or 
from dictaticta by the teacher or 

15. The final test fo^^ach lesson includes sen- 
tence dictation inlwjich the student must 
write the entire sernrence. 



16. The final test 
words or sente 
dictionary for 
translate to 



17. There are 
the studen 
meani nq&l 
phrasds or 







k ach lesson includes 
tten in phonetic or 
h the student must 
ar spelling. 



tivities in the lessons requiring 
rite spelling words in- some 
eative way such as making up 
tences or stories using the words 



18. The fcssons contain difficult words or demon, 
"^ds cfcJlttfcining "hard spots" in which the 
h i spotXs circled or underlined and the 
ient^^ asked to study the hard spot. 



Positive 
Scale 



Negative 
Scale 



19/ Lessons include words considered to be 
exceptions to the rule or generalization 
explained in the logons. 



NO 




'E 



20. There are apti 
quiring the stud 
regular spell 
[e.g. 9 sound 




the lessons ret- 
ranslate non- 
o regular spellings 
g, mixed-up letters]. 



21. There are haiWffriting activities that ask 
students tP prafctice forming letters/words. 



22. There 
practic 
ical mar 



23. Pr 





ivrties involving dictionary 
C\Tds alphabetizing , diacrit- 
s, etc. 



pns are made for students to copy 

are misspelled on the final lesson 
to take a test over those words at 
gome later. date. 

24. Provisions are made for the student to add 
w^rd^ to their spelling list that come from 
sources other than the spelling program it 
self. 



0 

NO 
NO 

YES 



YES 




YES 



SCORING PART II 

IIA. TOTAL WORDS CIRCLED ON EACH SCALE 
[Maximum Score = 24] 



POSITIVE NEGATIVE 



x 2 



IIB. TOTAL RATINGS 0 FOR STUDENT MATERIALS rjj" 
[Maximum Score = 48] ) I f W 



x 2 

m 



COMPOSITE SCORE, PART I AND II 

I IIA. Record Total Rating for the Teacher's 
Guide [Item IB : Maximum = 24] 



I I IB. Record Total Ratings For Student 
Materials p*€ffT I IB~T"Hax 7~sr-*s }- 



NEGATIVE 




x IIIC. TOTAL POSITIVE AND NEGATIVE 
\ RATINGS FOR GUIDE 6 MATERIALS 

\ [Maximum Score = 72] 



K Si 



ERIC 



